In today's world of technology, businesses are becoming increasingly dependent on the speed at which data can be processed. In the past, businesses were not able to acquire enough data to ensure that they were accurately and thoroughly informed. As the computer age blossomed, however, this problem has disappeared. Instead, businesses are now encountering the opposite problem; too much data is now available, and most of the data is irrelevant for any given purpose. Businesses are now encountering the problem of trying to determine what data is useful and what data is irrelevant.
With this influx of data, businesses have had to develop ways of making sense out of the mountains of data that are gathered. Event parsers are a common tool used for sifting through logged or streamed data to find useful and/or desired events in the data. An event parser works by examining incoming data and looking for known strings of information that are present in the type of data being sought. When a known string of data is found, the parser removes the known string, and/or a portion of data near the known string, and stores the removed data for analysis while ignoring or discarding the unwanted portions of the data. While parsers are an excellent way of finding useful data, they are somewhat cumbersome in that the speed at which they can analyze data is greatly affected by the configuration of the data to be analyzed and by the configuration of the parsers themselves.
In order to maximize the efficiency of the parsers, parser designers are often required to customize each parser for the specific application for which it is intended to be used, this can often involve travel to the site where the parser is used in order to configure the parser according to the configuration of the data to be analyzed. Such individual configurations can be costly for all of the parties involved as well as time consuming and inefficient. Furthermore, if the configuration of the data is changed, a technician is often required to reconfigure the parser to improve its efficiency.
Logged event data can be generated from a wide variety of sources, and each source is generally optimized and configured in a unique format depending upon the environment in which it is used. Parsers parse and normalize data (such as logged or streamed data) using methods and/or systems in which parse grammars are defined. Such grammars usually analyze event data by attempting to match events in the data to a specific predefined data expressions or definitions. As depicted in Prior Art FIG. 1, these expressions are commonly stored and compared in a hierarchical structure starting with the most specific grammars and progressing down the hierarchy through the less specific grammars towards the most generic grammars until a match is determined. Such systems can be very inefficient as data must be compared to the each expression until it is matched; if most of the data is matched to expressions not near the top of the hierarchy, time is wasted by comparing the data to the most specific grammars. Such schemes can also result in the loss of useful information if generic grammars are mistakenly placed above specific grammars in a hierarchy.
In some cases, only generic parse grammars are used, however, in these cases the derived meta data regarding each event is scanty and of limited value. Accordingly, in order to increase the usefulness of a parser it is beneficial to match as many events as possible to specific grammars. Configuring a parser to match specific grammars is a time consuming and error prone process.